7 research outputs found
Lasagna: Layered Score Distillation for Disentangled Object Relighting
Professional artists, photographers, and other visual content creators use
object relighting to establish their photo's desired effect. Unfortunately,
manual tools that allow relighting have a steep learning curve and are
difficult to master. Although generative editing methods now enable some forms
of image editing, relighting is still beyond today's capabilities; existing
methods struggle to keep other aspects of the image -- colors, shapes, and
textures -- consistent after the edit. We propose Lasagna, a method that
enables intuitive text-guided relighting control. Lasagna learns a lighting
prior by using score distillation sampling to distill the prior of a diffusion
model, which has been finetuned on synthetic relighting data. To train Lasagna,
we curate a new synthetic dataset ReLiT, which contains 3D object assets re-lit
from multiple light source locations. Despite training on synthetic images,
quantitative results show that Lasagna relights real-world images while
preserving other aspects of the input image, outperforming state-of-the-art
text-guided image editing methods. Lasagna enables realistic and controlled
results on natural images and digital art pieces and is preferred by humans
over other methods in over 91% of cases. Finally, we demonstrate the
versatility of our learning objective by extending it to allow colorization,
another form of image editing
ZeroWaste Dataset: Towards Deformable Object Segmentation in Extreme Clutter
Less than 35% of recyclable waste is being actually recycled in the US, which
leads to increased soil and sea pollution and is one of the major concerns of
environmental researchers as well as the common public. At the heart of the
problem are the inefficiencies of the waste sorting process (separating paper,
plastic, metal, glass, etc.) due to the extremely complex and cluttered nature
of the waste stream. Automated waste detection has great potential to enable
more efficient, reliable, and safe waste sorting practices, but it requires
label-efficient detection of deformable objects in extremely cluttered scenes.
This challenging computer vision task currently lacks suitable datasets or
methods in the available literature. In this paper, we take a step towards
computer-aided waste detection and present the first in-the-wild
industrial-grade waste detection and segmentation dataset, ZeroWaste. This
dataset contains over 1800 fully segmented video frames collected from a real
waste sorting plant along with waste material labels for training and
evaluation of the segmentation methods, as well as over 6000 unlabeled frames
that can be further used for semi-supervised and self-supervised learning
techniques, as well as frames of the conveyor belt before and after the sorting
process, comprising a novel setup that can be used for weakly-supervised
segmentation. Our experimental results demonstrate that state-of-the-art
segmentation methods struggle to correctly detect and classify target objects
which suggests the challenging nature of our proposed real-world task of
fine-grained object detection in cluttered scenes. We believe that ZeroWaste
will catalyze research in object detection and semantic segmentation in extreme
clutter as well as applications in the recycling domain.
Our project page can be found at http://ai.bu.edu/zerowaste/
VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting
Label-efficient and reliable semantic segmentation is essential for many
real-life applications, especially for industrial settings with high visual
diversity, such as waste sorting. In industrial waste sorting, one of the
biggest challenges is the extreme diversity of the input stream depending on
factors like the location of the sorting facility, the equipment available in
the facility, and the time of year, all of which significantly impact the
composition and visual appearance of the waste stream. These changes in the
data are called ``visual domains'', and label-efficient adaptation of models to
such domains is needed for successful semantic segmentation of industrial
waste. To test the abilities of computer vision models on this task, we present
the VisDA 2022 Challenge on Domain Adaptation for Industrial Waste Sorting. Our
challenge incorporates a fully-annotated waste sorting dataset, ZeroWaste,
collected from two real material recovery facilities in different locations and
seasons, as well as a novel procedurally generated synthetic waste sorting
dataset, SynthWaste. In this competition, we aim to answer two questions: 1)
can we leverage domain adaptation techniques to minimize the domain gap? and 2)
can synthetic data augmentation improve performance on this task and help adapt
to changing data distributions? The results of the competition show that
industrial waste detection poses a real domain adaptation problem, that domain
generalization techniques such as augmentations, ensembling, etc., improve the
overall performance on the unlabeled target domain examples, and that
leveraging synthetic data effectively remains an open problem. See
https://ai.bu.edu/visda-2022/Comment: Proceedings of Machine Learning Researc
Mortality and pulmonary complications in patients undergoing surgery with perioperative SARS-CoV-2 infection: an international cohort study
Background: The impact of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) on postoperative recovery needs to be understood to inform clinical decision making during and after the COVID-19 pandemic. This study reports 30-day mortality and pulmonary complication rates in patients with perioperative SARS-CoV-2 infection. Methods: This international, multicentre, cohort study at 235 hospitals in 24 countries included all patients undergoing surgery who had SARS-CoV-2 infection confirmed within 7 days before or 30 days after surgery. The primary outcome measure was 30-day postoperative mortality and was assessed in all enrolled patients. The main secondary outcome measure was pulmonary complications, defined as pneumonia, acute respiratory distress syndrome, or unexpected postoperative ventilation. Findings: This analysis includes 1128 patients who had surgery between Jan 1 and March 31, 2020, of whom 835 (74·0%) had emergency surgery and 280 (24·8%) had elective surgery. SARS-CoV-2 infection was confirmed preoperatively in 294 (26·1%) patients. 30-day mortality was 23·8% (268 of 1128). Pulmonary complications occurred in 577 (51·2%) of 1128 patients; 30-day mortality in these patients was 38·0% (219 of 577), accounting for 81·7% (219 of 268) of all deaths. In adjusted analyses, 30-day mortality was associated with male sex (odds ratio 1·75 [95% CI 1·28–2·40], p\textless0·0001), age 70 years or older versus younger than 70 years (2·30 [1·65–3·22], p\textless0·0001), American Society of Anesthesiologists grades 3–5 versus grades 1–2 (2·35 [1·57–3·53], p\textless0·0001), malignant versus benign or obstetric diagnosis (1·55 [1·01–2·39], p=0·046), emergency versus elective surgery (1·67 [1·06–2·63], p=0·026), and major versus minor surgery (1·52 [1·01–2·31], p=0·047). Interpretation: Postoperative pulmonary complications occur in half of patients with perioperative SARS-CoV-2 infection and are associated with high mortality. Thresholds for surgery during the COVID-19 pandemic should be higher than during normal practice, particularly in men aged 70 years and older. Consideration should be given for postponing non-urgent procedures and promoting non-operative treatment to delay or avoid the need for surgery. Funding: National Institute for Health Research (NIHR), Association of Coloproctology of Great Britain and Ireland, Bowel and Cancer Research, Bowel Disease Research Foundation, Association of Upper Gastrointestinal Surgeons, British Association of Surgical Oncology, British Gynaecological Cancer Society, European Society of Coloproctology, NIHR Academy, Sarcoma UK, Vascular Society for Great Britain and Ireland, and Yorkshire Cancer Research